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fy | ■ We aim at characterizing viability, invariance and some reachability properties of controlled 

piecewise deterministic Markov processes (PDMPs). Using analytical methods from the theory 
of viscosity solutions, we establish criteria for viability and invariance in terms of the first order 
normal cone. We also investigate reachability of arbitrary open sets. The method is based on 
viscosity techniques and duality for some associated linearized problem. The theoretical results 
are applied to general On/Off systems, Cook's model for haploinssuficiency, and a stochastic 
■ model for bacteriophage A. 
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Markov processes have been intensively used to describe variability features of various cellular 



1 Introduction 

> 

\ processes. To our best knowledge, Markovian tools have first been employed in connection to 

molecular biology in |15j . The natural idea was to associate to each reaction network a pure jump 
model. Due to the large number of molecular species involved in the reactions, direct simulation of 
these models turns out to be very slow. To increase proficiency, hybrid models are adopted in |13j . 
They distinguish the discrete components from the "continuous" ones. Using partial Kramers- 
Moyal expansion, the authors of [13] replace the initial pure jump process with an appropriate 
piecewise deterministic Markov one. 

One may reduce the complexity of PDMPs by restricting the model to some invariant set 
containing the initial data, whenever this is known. Compact invariant sets are also needed for effi- 
ciently implementing algorithms. Another important issue that can be approached using invariance 
are the stable points. In particular, a fixed point for which one finds arbitrarily small surrounding 
invariant sets is stable in the sense of Lyapunov. 

We begin by characterizing e- viability of controlled PDMPs via some associated control problem. 
A closed set of constraints K is said to be viable (or e-viable) with respect to some dynamic 
control system if, starting from K, one is able to find suitable controls keeping the trajectory 
in K (or, at least in some arbitrarily small neighborhood of the set of constraints). Viability 
properties have been extensively studied in both deterministic and stochastic settings (for Brownian 
diffusions), starting from the pioneer work of Nagumo. The methods used to describe this property 
for deterministic or diffusion processes rely either on the Bouligand-Severi contingent cone (cf. [2J, 
[3], [16]) or on viscosity solutions ([I], [5], [6], [7], [H]). Using analytical tools from viscosity theory, 
we provide a geometrical characterization of e-viability and invariance of some set of constraints 
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K with respect to the controlled piecewise deterministic Markov process. As for the Brownian 
diffusion case (cf. [7]), the criterion involves the normal cone to the set of constraints and is 
completely deterministic. Similar arguments allow one to characterize the invariance of the set 
of constraints. We emphasize that these geometrical conditions can be rather easily checked for 
PDMPs associated to gene networks. In order to illustrate these theoretical assertions, two examples 
are considered. For general On/Off models, we show how the invariance criterion can be used in 
order to reduce the state space to a compact set. We also characterize points that can be chosen as 
candidates for stability (in the sense that one finds arbitrarily small surrounding regions that are 
invariant). Another biological example is a model for bacteriophage A (described in p2]). Although 
it is more complex, one can still use the invariance criterion to characterize candidates for stability. 
It turns out that only one such point exists in the absence of impulsive exterior control factors. 

The second aim of the paper is to characterize the reachability property of arbitrary open sets 
with respect to the controlled piecewise deterministic Markov process. The criterion is obtained 
using viscosity methods. Recently, the paper [10] has provided a linear programming formulation 
for discounted control problems in the framework of SDEs driven by standard Brownian motion. 
The reachability problem can be connected to the value function of some appropriate piecewise de- 
terministic control system. Using the idea in [TO], we give a criterion involving the dual formulation 
of the linearized version of the initial problem. To illustrate this result, we consider Cook's model 
for haploinsufnciency introduced in [12]. Our criterion allows one to prove that, starting from any 
arbitrary point, one reaches any arbitrarily given open region, with positive probability. 

The paper is organized as follows: In Subsection 11.11 we briefly recall the construction of con- 
trolled PDMPs and state the main (standard) assumptions. Section [5] is devoted to the study of 
viability property (Subsection 12. ip and invariance (Subsection I2.2p with respect to the PDMP. The 
criteria involve the normal cone to the set of constraints and the characteristics of the process. 
Section [3] deals with the reachability property. We use a Krylov-type argument to provide some 
dual formulation of the associated control problem. In Subsection 14.11 we recall some rudiments 
on the PDMPs associated to a system of chemical reactions. We consider two biological examples: 
the On/Off model (Subsection 14. 2p and the bacteriophage A (Subsection I4.3p . We first study the 
compact invariant sets for the On/Off model. For a particular case (the so-called Cook model for 
haploinsufnciency) , we prove that every open set can be reached with positive probability, starting 
from any initial point. In the case of the bacteriophage A (described in [T7]), our invariance criterion 
allow to identify the stable point of the system. The Appendix provides the comparison principle 
and some stability results for viscosity solutions. 



1.1 Construction of controlled PDMPs and main assumptions 

We let U be a compact metric space (the control space) and K w be the state space, for some N > 1. 

Piecewise deterministic control processes have been introduced by Davis [14] . Such processes 
are given by their local characteristics: a vector field / : l w x U — > R N that determines the 
motion between two consecutive jumps, a jump rate A : R N x U — > R+ and a transition measure 
Q : R N x U x B (R N ) V {R N ) . Here B (R N ) is the family of Borel sets and V (R N ) stands for 
the family of probability measures on R N . For every A G B (R N *) , the function (u, x) \— > Q (x, u, A) 
should be measurable and, for every (x, u) G R N x U , Q (x, u, {x}) = 0. 

We summarize the construction of controlled piecewise deterministic Markov processes (PDMP). 
Whenever u G L° (K^ x R+; U) (u is a Borel measurable function) and (to,xo) G R+ x R , we 
consider the ordinary differential equation 



tQ,XQ,U 
t 



tQ,XQ,U 

to 



to,Xo,U 

t ■ 



U ( 



(x Q ,t- t ) )dt, t> t , 
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We choose the first jump time T% such that the jump rate be A l^' xo,u , u (xo,t) 

P (Ti >t) = exp (- J A ($°>*°> u , u (x , s)) cis^ . 

The controlled piecewise deterministic Markov processes (PDMP) is defined by 

x x ,u = $a,x ,u^ i{te [0)Ti) _ 

The post-jump location Y\ has Q f^' x °' u , u (xq> r) , ■) as conditional distribution given T\ = r. 
Starting from Y\ at time T%, we select the inter-jump time T 2 — T\ such that 



' (T 2 - Ti > t I T\ y Y\) = exp (- jf * A u (Fi, s - H)) ds) . 



We set 



x x ,u = qT 1 ,y 1 ,u^ ift€ [ Tl)T2 )_ 



The post-jump location Y2 satisfies 



P (y 2 G ^ / Ti,^) = Q [$tI' Yi ' u , u (Yi,T 2 - Ti) ,i4 

for all Borel set A C R^. And so on. 

Throughout the paper, unless stated otherwise, we assume the following: 

(Al) The function / : ~ML N x U — > R w is uniformly continuous on R w x U and there exists a 
positive real constant C > such that 

(Al) \f(x,u)-f(y,u)\<C\x-y\, and \f(x,u)\<C, 

for all x, y G R^ and all u £ U. 

(A2) The function A : R^ x J7 — >• R+ is uniformly continuous on W N x {7 and there exists a 
positive real constant C > such that 



(A2) 



I A (x, it) — A (y,u)\ < C \x — y\ , and A (x, u) < C, 



for all 1,1/6 R^ and all u G {/. 

(A3) For each bounded uniformly continuous function /i G £?[/ C (R^) , there exists a continuous 
function rjh : R — s- R such that % (0) = and 



(A3) 



sup 



h (z) Q (x, it, dz) 



h (z) Q (y, u, dz) 



< Vh (\x - y\) . 



(A4) For every x G R^ and every decreasing sequence (r n ) n>0 of subsets of R^, 



(A4) 



inf sup Q (x, u, r n ) = sup Q ( x, u, nT n 

n>0 ue u ueU 



Remark 1 We have kept A3 as it appears in Soner flftf . However, this assumption may be some- 
what weakened by imposing 

(A3') For each bounded uniformly continuous function h G BUC (R^) , there exists a continu- 
ous function rjh ■ R — > R such that rjh (0) = and 



sup 

ueu 



A(x,u) / h(z) Q (x,u,dz) — X(y,u) / h(z) Q (y,u,dz) 



< rjh (\x - y\) . 



It is obvious that whenever one assumes A3 and A (•) is bounded, the assumption A3' holds true. 
Moreover, all the proofs in this paper can be obtained (with minor changes) when A3' replaces A3. 
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2 A geometric condition for viability and invariance 



2.1 Conditions for viability 

This subsection aims at characterizing the viability property of a nonempty, closed set K C M. N . In 
analogy to the deterministic framework, this property is proved to be connected to some geometric 
condition involving the normal cone to K. The proof of the characterization relies on the viscosity 
solution concept. We begin the subsection by recalling the notions of viability (respectively e- 
viability) and normal cone. 

Definition 2 1. A nonempty, closed set K C ~R. N is said to be viable with respect to the controlled 
piecewise deterministic Markov process X if, for every initial point x £ K , there exists an admissible 
control process u G L° (R N x M. + ; U) such that X^ ,u G K, F '-almost surely, for all t > 0. 

2. A nonempty, closed set K C M. N is said to be e-viable with respect to the controlled piecewise 
deterministic process X if, for every initial point x G K and every e > 0, there exists an admissible 
control process u £ G L° CR. N x M + ; U) such that 



E 



£V (d K [xr £ ) ai 



dt 



< e. 



Here, dx stands for the distance function to the closed set K . 

Definition 3 Let K C be a closed subset and let x be a point of K. The normal cone to K at 
x, denoted by Nk (x), is defined as 

N K (x) = {p G R N : Ve > 0, 3n > such that My G K DB (x,rj) , (p, y - x) < e \y - x\} . 

We recall that B (x, rf) = {y G ~R. N : \y — x\ < r/} . 

The definition of the e-viability property of a nonempty, closed set K C can, alternatively, 
be given with respect to the value function 



(1) v(x) = inf E 

«eL (M Jv xR + ;[/) 



oc 



e-* (d K (X?' u ) A 1) dt 

for all x G M iV . Indeed, with this notation, the set K is e-viable if and only if the restriction of v to 
K is zero. We consider the associated Hamilton-Jacobi integro-differential equation 

(2) v (x) - d K (x) A 1 + H (x, Vv (x) , v) = 0, 

for all x G M. N , where the Hamiltonian is given by 



(3) H (x, p, ip) = sup < - (/ (x, u) , p) - X (x, u) I O (z) -ip{x))Q (x, u, dz) 

ueu I Jw 1 

Under the assumptions (A1)-(A3), the function v is known to satisfy (cf. [18] . Theorem 1.1), in 
the viscosity sense, Equation ([2]). We are going to need a slightly more general definition for the 
viscosity subsolution (respectively supersolution) then the one used in [IB] . 

Definition 4 A bounded, upper (lower) semicontinuous function v is a viscosity subsolution (su- 
persolution) of (0|j) if, for any test-function (p G (N x ), on some neighborhood N x of x G M. N , 
whenever x is a maximum (minimum) point of v — <p, 

v(x) - d K (x) A 1 + H (x, V> (x) , v) <0 (> 0). 

A bounded, continuous function v is a viscosity solution of (0) if it is both subsolution and 
supersolution. 
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At this point, we introduce a technical assumption on the transition measure Q which provides 
a comparison principle. It states that the probability for the post jump position to be arbitrarily 
far away from the pre jump one is uniformly small. We emphasize that this assumption is made in 
order to give a simple proof for the comparison principle. However, it is not essential; one can, as 
an alternative, strengthen A3 as in [T] Section 3. Moreover, the main results of the paper hold true 
independently of this assumption, whenever a comparison principle for semicontinuous functions 
holds true. 

(A5) We assume that 

(A5) inf sup Q (x,u,R N \ B(x,n)) = 0. 

Remark 5 Assumption (A5) is obviously satisfied if Q does not depend on x and u. Moreover, all 
the piecewise deterministic processes associated to chemical reactions (see Subsection \4-l\ ) satisfy 
(A5). 

Proposition 6 (Comparison Principle) Let W be a bounded u.s.c. viscosity subsolution of (0j 
and let V be a bounded l.s.c. viscosity supersolution of fSJJ. Moreover, we assume that either W or 
V is uniformly continuous. Then 

W(x) <V{x), 

for all x G R N . 

The arguments for the Proof are standard. For reader's convenience, we give the Proof in the 
Appendix. 

The main result of the subsection is the following characterization of the e-viability property 
with respect to the controlled piecewise deterministic Markov process. 

Theorem 7 Given a nonempty, closed set K C R , the following properties are equivalent: 

(i) K is e-viable; 

(ii) The following assertions hold simultaneously: 

(a) for every x G dK, and every p G Nk (x) , 

inf {(/ (x, u) , p) + A (x, u) Q (x, u, K c )} < 0. 

o 

(b) for every x £ K, 

inf {X(x,u)Q(x,u,K c )} < 0. 
Proof. We begin by proving that (ii) =^> (i). We claim that the function 

^ ^ | 1, otherwise. 

is a viscosity supersolution for ([2]). By definition, V is lower semi-continuous. Obviously, the 
supersolution condition holds true for all x G R N \ dK. Let us now fix a point x G dK. If 
(/)£Cj [M x ) , for some J\f x C R N neighborhood of x, is such that (V — (f) admits a global minimum 
at x, then Vip (x) G Nk (x). Thus, the condition (ii) yields 

V (x) - (d K (x) Al) + H (x, Vip (x) , V) = - inf {(/ (x, u) , Vip (x)} + A fx, u) Q (x, u, K c )\ > 0. 

ueu 

It follows that V is a bounded viscosity supersolution for ([2]). Using the Comparison Principle, we 

get 

v(x) < V(x) = 0, 
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for all x E K and the e-viability of K follows. 

To prove the converse, we introduce, for every m E N*, the value function v m , defined by 



Vm(x) = mv x 



inf E 

u£L,°(R N xR+;U) 



me"* (d K (X?> u ) A 1) dt 



for all x E R . Then, Theorem 1.1 in [18] yields that v m is the unique bounded viscosity solution 
of 



(4) 



v m (x) - m (d K (x) Al) + H (x, Vv m (x) , v m ) = 0, 



where the Hamiltonian H is given by ([3]). 

Step 1. We claim that there exists a positive constant c > such that, for all x E K c and all 
m > 1, 

(5) 



% (x) > mc (cZr- (x) A 1) . 



We recall that on the set {T\ > t} , Xf' u = §®> x > u . Standard estimates yield the existence of a 
positive constant c\ which is independent of x, u and t such that 



<Pj — X 



< Clt, 



for all t > 0. Thus, on the set |Ti > ^|c^~ } one S e ^ s 

d K {XT) a i > > o, 

for all s < djf (^) A1 . Using the Assumptions A1-A2, one easily proves that 



2ci 



E 



me"* (djf A 1) dt 



> mE 



— e _ t d K (x)Al dti 



> Cm (d K (x) A 1) 



Hence, (0 holds true for all x E K c . 

Step 2. Let us fix x E (9-ftT. We consider an arbitrary p E Nr (x) and introduce the test function 

1 2 

WW = (P,y~x) -m* |y-ac| , 
for all i/ G R w . We let x m £ B (x, 2) such that 

(6) v m (x m ) - (p (x m ) < v m (y) - (p (y) , 

for all y £ -B (x, 2). One notices that, for large enough m, x m £ B (x, 1). Indeed, this is a simple 
consequence of the fact that v m (x) = ip (x) = and, thus, 

(7) < V m [Xm) < V {Xm) < (p,x m ~ x) - m* \x m - x\ . 

Moreover, for large enough m, the inequality © holds true for all y £ R . The inequalities ([5]) and 
© yield 



mc (d K (x m ) A 1) <v m (x m ) < (p, x m - x) - m± \x m - x\ 



This implies 



(8) lim m (die (x m ) A l) 2 = 0, lim x m = x and lim m<* \x m — x\ 2 = 0, 

m— too m— ¥oo m— >oo 
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and 

(9) lim v m (x m ) = 0. 
We claim that 

(10) limsupm* \x m — x\ = 0. 

m— >oo 

We assume that, on the contrary, there exists some positive real constant 8 > such that 

(11) 771,4 \x m — x\ > 6, 

for all m > 1. For every m > 1, we choose some y m £ K such that 

(12) d K (x m ) = \x m - y m \ . 

The equalities ([8j) imply that lim y m = x. Together with the choice of p £ Nk (x) , the last limit 
yields 

(13) (p, J/ m - < « ~ x l ' 

for every m large enough. To simplify the notation, we assume that (|13p holds true for all m > 1. 
Using the inequalities ((7J), (fT3|) and (fTTj) . we have 

- 2 

< (p, x m — x) — nit \x m — x\ 

< (P, Vm ~ x) + (p, x m - y m ) - m~i \x m - x\ (\y m - x\ - \x m - y m \) 

S i 

< 2 IVm ~ x \ + IpI d-K (x m ) - 5\y m - x\+ m*d K (x m ) \x m - x\ . 



Therefore, 



5 < mi \x m - x\ < mi (\y m - x\ + d K (x m )) 



(2 \ i 2 i 

(14) < I - |p| + 1 I mid K (x m ) + -m^d K (x m ) \x m - x\ . 

We allow m — > oo in the inequality (|14p and recall that © holds true to come to a contradiction. 
It follows that ()10p must hold true. 

We recall that the function v m is a bounded, continuous viscosity supersolution of (|3J) to get 

(x m ) - m (rfx (x m ) A 1) 

> inf {(p,/(^.))-»»i^-x,/(*.,«» + A(*..-) / (» m W-»„(x m ))0(, n ,, U ,^) 
Assumption Al yieds 
(P,f(x,u)) 

= (P, f {x m , u)) - 2mi (x m -x,f (x m , u)) + (p, f (x, u) - f {x m , u)) + 2mi (x m -x,f (x m , it)) 
(15) 

< (p, / (a^m, «)) - 2m4 (x m -x,f (x m , u)} + C \p\ \x m - x\ + Cm* |x m - x| , 
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for all m > 1 and all u E U. Here C is a generic real positive constant that is independent of m > 1 
and u £ U and may change from one line to another. Let us fix mo > 1. Then, for all m > mo and 
all u E £/, by Assumptions A2-A3, we obtain 

A(sc m ,it) / mc (<2jr (z) A l) 2 Q (x m ,u, dz) — A (x m , u) v m (x m ) 



> cm A (x, u) I (d K (z) A l) 2 Q (x m , u, dz) - C \x m - x\ m - Cv m (x m ) 

> cm A (x, u) / (d K (z) A l) 2 Q (x, u, dz) - m Cr] (d 1)2 (\x m - x\) - C \x m - x\ m - Cv m (x m ) 
(16) 

> A (x,u) Q (x,u,K c ) - CQ (x,u,K mo ) - moCrj^^ (\x m - x\) -C\x m - x\ m - Cv m (x m ) , 
where we use the notation 

1 



K mo = ^zeK^.d K (z)< ^_ 

Finally, using (fT5|) and (JT6J) , we get 

{(p, / (x, u)) + A (x, u) Q (x, u, K c )} 

< (p, / (x m , u)) - 2m? (x m - x, f (x m , u)) + C \p\ \x m -x\+ Cm* \x m - x\ 
+ A( Xmi U ) I (»m (^) - (Sm)) Q ( dz) + C sup Q(x,u, K mo ) 



(17) + m Q Cr]( dKA1 ^ (\x m - x\) + C \x m - x\ m + Cu m (x m ) , 

for all m > mo and all u £ U. We take in (|17p the infimum over u £ U, then limsup as m — > oo 
and recall that the inequalities (JS]), Q, (fTUj) hold true, to have 

(18) inf {(p, f (x,u)) + A (x, u) Q (x,u,K c )} < Csup Q (x,u,K mo ) 

for all mo > 1. Notice that (K mo ) is a decreasing sequence of sets such that ri mo >iK mo = <f>. Then, 
using the Assumption A4, the inequality (fT8"|) yields 



(19) inf {(p, / (x, «)) + A (z, «) Q (x, n, i^ c )} < 0. 

Step 3. For x E K, we take the test function 99 (y) = — \y — x\ 2 for all y G R^. The same 
arguments as in Step 2 give 

(20) inf {A(x,u)Q(x,n,if c )} < 0. 



2.2 Conditions for invariance 

Another problem, closely related to viability is the invariance of a nonempty, closed set K C M. N . 
Whenever this property is satisfied, the controlled PDMP remains in K independently on the 
control process and as soon as the initial datum x G K. Suppose that the initial states of the model 
to which the PDMP is associated are known. Then, one may reduce the complexity by restricting 
the model to some invariant set containing the initial data. We begin by recalling the notion of 
invariance. 
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Definition 8 A nonempty, closed set K C l w is said to be (strongly) invariant with respect to 
the piecewise deterministic Markov process X if, for every initial point x £ K and every admissible 
control process u, X^' M G K, ¥ '-almost surely, for all t G R+. 

The invariance property is related to an optimal control problem for which the value function 
v inv is given by 

oo 

e" 1 (d K (X?> u ) A 1) dt 



o 



(21) v inv (x) = sup E 

u£lP(R N xM.+;U) 

for all x G R N . The main result of the section is 

Theorem 9 Let K C 1^ be a nonempty, closed subset. The following statements are equivalent: 

(i) The set K is invariant; 

(ii) The value function V{ nv (x) = 0, for all x G K. 
(Hi) The following conditions hold simultaneously: 

(a) for every x G dK, every p G Njc (x) , and every u G U, 

{f (x, u),p)+X (x, u) Q (x, u, K c ) < 0. 

o 

(b) for every x G K,and every u G U, 

X(x,u)Q(x,u,K c ) < 0. 

Proof. We only need to prove that (ii) and (Hi) are equivalent. We begin by proving that (Hi) 
implies (ii). By Theorem 1.1 in [18], the function 

is the unique bounded viscosity solution of the Hamilton-Jacobi integro-differential equation 

(22) w (x) + d K (x) A 1 + H (x, Vw, w) = 0, 

where the Hamiltonian is given by ([3]) . As in the proof of Theorem [71 one notices that the function 

V(x) = -l K c(x), for all x G Fl N 
is a viscosity subsolution of (|22|) . By the comparison principle, we get that 

Vinv (x) < 0, 

for all x G K. The statement follows. The proof of the converse relies on the same arguments as 
Steps 1-3 of Theorem [JJ ■ 

3 Reachability of open sets 

Stability issues are very important for biological networks. For deterministic models, one can easily 
decide whether the system is stable, bistable, etc. However, the behavior is much less obvious 
for a piecewise deterministic approach. One should expect that the trajectories of the controlled 
PDMP starting from some region around the stable point converge to it. Alternatively, a point for 
which arbitrarily small surrounding regions are invariant (or at least viable) is a good candidate 
for stability. Thus, the issue of stability may be addressed via viability techniques. In the case of 
multiple stable points, given an arbitrary initial state, it would be interesting to know to which of 
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these regions the trajectories of the PDMP are directed. The goal of this section is to address the 
problem of reachability. 

Let us consider an arbitrary nonempty, open set O C R N . As in the case of viability, the 
techniques we use rely on the theory of viscosity solutions for a class of Hamilton-Jacobi integro- 
differential equations. We are going to introduce a slight difference in our coefficients allowing to 
consider a control couple. To this purpose, we make the following notations: We let the vector field 
/ : R N x U x B (0, 1) — > R N be given by 

(23) f(x,u 1 ,u 2 )=f(x + u 2 ,u 1 ), 

for all x G R N , u 1 G U and u 2 G B (0, 1) . Similarly, the function X : R N x U xB (0,1) — > R + is 
given by 

(24) A (x, u l ,u 2 ) = A (x + u 2 ,u r ) , 
and 

Q (x,u l ,u 2 , A) =Q[x + u 2 ,u l ,A + u 2 ) , 
where A + u 2 = [a + u 2 : a G A] , for all x G R N , u 1 £U, u 2 £B (0, 1) and all Borel set A C R N . 

Remark 10 1. It is obvious that, for every h€C b (R N ) and every x G R N , u 1 £U,u 2 G B (0, 1) , 

h (z) Q (x, u, u 2 , dz) = I h (z — u 2 ) Q [x + u 2 ,u l ,dz) . 

2. One can easily check that the assumptions (Al)-(A2) and (A5) hold true for the characteristic 
f,X,Q) replacing (/, A, Q) and the set of control U replaced by U x B (0, 1) . 



Throughout the section we are going to strengthen (A3) and assume 

(B) For each bounded uniformly continuous function h G BUC (M. N ) , there exists a continuous 
function rj^ : R — > R such that r)h (0) = and 
(B) 

h{z — n 2 ) Q (x + u 2 , u 1 , dz) — \ h(z — u 2 ) Q (y + u 2 ,u, dz) 



sup 



< 



■rih \\x - y\ 



Remark 11 Similarly to RemarkUl one can alternatively assume 

(B') For each bounded uniformly continuous function h G BUC (R ) , there exists a continuous 
function rjh '■ R — > R such that rjh (0) = and 



sup 



X [x + u 2 , u 1 ) J RN h(z — u 2 ) Q (x + u 2 , u 1 ,dz) 
-X (y + u 2 , u 1 ) J RJV h [z — u 2 ) Q (y + u 2 ,u, dz) 

< Vh (\x - y\) ■ 



For every e > 0, we denote by £ 6 the class of measurable processes u : R xR + — > B (0, e) . For 

1 2 

every admissible control couple (u , u 2 ) G L° (R N x R+;U) x£ e , we let X x ' u ' be the piecewise 
deterministic process associated to the characteristic A, Q^j . Obviously, X x ' u '° is associated to 
(/,A,Q). 



Definition 12 Given an initial condition x G O c (or even x G R ), the set O is reachable starting 
from x if there exists some admissible control process u such that the set 

jx?' u '°G0, te [0 >O o)} 

has positive probability. 
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In connection to this property, we define, for every e > 0, the value function 



(25) 
for all x £R N 



v £ (x) = inf E 

u 1 GL° (R N xl + ;U),u 2 e£ s 



e-Udoc [XI 



Remark 13 It is obvious that, whenever v°(x) = 0, the set O is not reachable starting from the 
point x. On the other hand, whenever v® (x) < 0, there exist a constant 6 > 0, an admissible control 

process u G L° (R N x R+;U) and T > such that E ^ V* (do* fxf" ' ) Al)du > 5. It 

follows that the set must have positive probability. Thus, O is reachable 

from x if and only if v°(x) < 0. 

Theorem 1.1 in Soner [18] yields that v £ is the unique bounded viscosity solution of the following 
Hamilton-Jacobi integro-differential equation: 

= v e (x) + sup <^ do? (x + u 2 ) A 1 + sup {- (/ [x + u 2 , u 1 ) , Vv £ (x)) 
\u 2 \<£ [ vleu 



X(x + u 2 ,u 1 ) / (v £ (z) - v £ {x))Q (x,u 1 ,u 2 ,dz) 



(26) 



for all x G R N . For the particular case e = 0, the value function v° is the unique bounded uniformly 
continuous viscosity solution of 



(27) 

for all x £ 



v° (x) + d c (x) A 1 + H (x, v (x) , v°) = 0, 
where the Hamiltonian H is given by ©• 



Remark 14 As a consequence of the definition of Q, for every e > and every u 2 G B (0, e), the 
function w (•) = v £ (• — n 2 ) is a viscosity subsolution of [27\ ). 



We get the following convergence theorem 

Theorem 15 There exists a decreasing function n : 
such that 



that satisfies lim e _^o V ( £ ) = and 



(28) 



sup |v e (x) — v (x) I < rj (e) 

xeR N 



Proof. We recall that v° is uniformly continuous and let 

(29) w°(r) = sup{\v°{x) -v°{y)\ : x, y G R N , \x - y\ < r) 



for all r > be its continuity modulus. Let us fix x G R N and e > 0. We denote by $ to ' Xo,u ,u 
the flow associated to the vector field /. Standard estimates and the assumption (Al) yield the 
existence of some positive constant C > which is independent of x and e > such that 



(30) 



St 



< Ce, 



for all t G [0, 1] , and all (n 1 , n 2 ) G L° (R N x M + ; [/) x £ £ . The constant C is generic and may change 
from one line to another. We emphasize that throughout the proof, C may be chosen independent 



of x G R N , e > and of {u l ,u 2 ) G L° 



. ; U) x £ 6 . Using the dynamic programming 
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principle (Soner [18J, Equation (0.8)), for every admissible control process u G L ( 



1 c ¥ {juiN 



-;U), 



the following inequality holds true 

rTiAl 



(31) 



■u (x) < E 



X?" l > Q ) Al)dt + e' T ' M v° (X^f 



We consider an arbitrary admissible control couple (u , u 2 ) G L° (K^ xR + ;[/)x^. For simplicity, 
we introduce the following notations: 



A 1 (t) = A (d^" 1 ' , n, 1 ) , A 1 (t) = exp (- jf* A 1 



A 1 '^ (t) = A ($^' w •« + uf , u]) , A 1 ' 2 (t) = exp I - j| A^ (,) d S j , 

for all i > 0. We denote the right-hand member of the inequality (|3ip by /. Then, / is explicitly 
given by 

/ - / 1 A^A 1 (t) C -e- s (d c (qQm 1 ' ^ a l) dsdt 



+ \ 1 (t)A 1 (t)e- t v°(z)Q(^ ulfi ,u 1 t ,dz)dt 

JO JRN \ ) 

+ A 1 (1) £ -e-* (d c (s^ 1 ' ) Al)dH A 1 (1) e~ V ^J*." 1 - 



= Ii + J 2 + 1 3 + / 4 . 
Using the inequality (f3"D"l) and the assumption (A2), one gets 



(32) 



(33) 



h < X X ' 2 (t)A X ' 2 (t) J* -e- s (d c ($f ul > u2 + uf) A l) dsdt + Cs, 



h < A ' (1) / -e- 1 ( doc ( ) A 1 ) dt + Ce. 



For the term I2, one has 
h < f X^WA^We-* 







f 2 — u y 



$^ u1 '" 2 + u%, u\, dz) dt + C(e + W v o (Ce) + u° (e)) 



< 



f^WA^We"' / ^^Qf^.ul,^,^ dt + C(e + W„o (Ce) + u; (e)) 



(34) 



+ ( f X x ' 2 (t)A x ' 2 (t) e-'dt] sup \v°(z) - v £ (z)\ . 
Finally, 

h < A^ 2 {l)e' l v° (s^' 1 * 3 ) + C (co° (Ce) + e) 
(35) < A^ClJe-V ($f' ul ' u2 ) + C (w° (Ce) + e) + A l > 2 {l)e~ l sup (u (z) - v e (z)) . 

We substitute ([32])-® inpijl. We take the infimum over the family of (it 1 , u 2 ) G L° (R^ xl + ;(/) 
£ £ and use the dynamic programming principle to have 

v° (x) < v £ (x) + C(e + W v o (Ce) + w° (Ce)) 

+ (J A 1 ' 2 (t)A 1 ' 2 (t) e~*dt + A 1 ' 2 (1) e- 1 ^ sup - v £ (z)) . 
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We notice that 
-I 



A 1 ' 2 ^ 1 ' 2 (i) e~*dt + A 1 ' 2 (1) e" 1 = 1 



o 

Thus, 



•Jo*** 



e~*(fi < 1 



-(A max +1) 



v° (x) - v £ {x) <C(e + W v0 (Ce) + w (Ce)) + (l - e -(* max +i)) sup («°(*) - w £ (z)) . 

The conclusion follows by taking the supremum over x G R and recalling that C is independent 
of x and e > 0. ■ 

We introduce the function [i* : M. N — > R defined by 



(36) 
where 



H* (x) = sup lip G C£ (R w ) such that V (y, u) G 

/i < W> (y) - (d 0c (y) A 1) + (<p (x) - if (y))} , 



x U, 



(37) «<V (y) = (y) , / (y, «)) + A (y, u) / (2) - p (y)) Q (y, u, cte) 



for all y € R . This function is inspired by the results in [10] , It corresponds to the dual form 
of some linearized formulation for the discounted control problem. In fact, one can interpret the 
initial problem by using occupational measures. In a second step, the set of occupational measures 
can be enlarged to a set of measures satisfying appropriate conditions. These conditions involve the 
infinitesimal generator of the underlying process and can be interpreted as a classical constraint. 
Minimizing on this set leads to the same value function. Duality techniques then allow to give a 
formulation much like /x* (but for generators associated to Brownian diffusion processes). 

The main result of the section gives the equality between the reachability value function v° and 

M • 

Theorem 16 For every x G M. N , the equality 

(38) v°(x) = ff(x) 
holds true. 

Proof. We begin by proving that 

(39) v°(x) > fi*(x), 

for all x G R N . We fix x G R^ and (p, y) G R x C\ (R^) such that 

li < U u <p (y) - (d c (y) A 1) + (if (x) - if (y)) , 



for all y G M N , u G U. Then, for every mGL 1 



(m>N 



x,u,0 



A 1 



for all t > 0. Using Ito's formula (cf. Theorem 31.3 in [2]), the last inequality yields 



u < lim E 

T->oo 



+ ip(x) + E 



x,u,0 



dt 



-e~ l (d c (x*' u '°) A l) dt 



(40) 



lim e~ T E 
T->oo 



E 



if \X T ' 



+ E 

* [d c (Xf* ) A l) 



e~ l (d c (XI 



M\dt 



dt 
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for all T > 0. We recall the definition (f3U|) of p* (x) and the inequality (f3T?j) follows from ([10|). 
In order to complete the proof of the Theorem, we still have to prove that 

(41) n*(x) > v°(x). 

Let us consider (p e ) a sequence of standard mollifiers p £ (y) = -pj-p (|) , y G R , e > 0, where 
p G C°° (R^) is a positive function such that 



Supp(p) C (0, 1) and / p{x)dx = 1. 



We introduce the functions 



(42) = * Pe , 

for all e > 0. We claim that these functions are (viscosity) subsolutions of (|27|) . The Proof follows 
the same arguments as Lemma 2.7 in Barles, Jakobsen [9]. For convenience, we give the Proof in 
the Appendix. Using the fact that V s is a subsolution of flZTJ) , one gets 

U e (x) < p* (x) . 

It follows, from ([28]) that 



(v° * p e ) (x) < /i* (l) + V 0) • 

We allow e — >■ in the last inequality, and recall that v° is continuous, to finally get (|4ip . The 
Proof of the Theorem is now complete. ■ 

The previous result gives the following interesting characterization of the reachability of the set 

O : 



Criterion 17 Let x G R^ be an arbitrary initial state. Then the controlled piecewise deterministic 
Markov process starting from x reaches O if and only if there exists n G N* such that for every 
if G Cl (R N ) there exists u£U,y G R^ such that 



(43) U u <p (y) - d c (y)M + & (x) - up (y)) < -n 



4 Biological examples 

4.1 Biochemical reactions and mathematical assumptions 

We begin by recalling some rudiments on piecewise deterministic Markov processes associated to 
gene networks. For further contributions on gene networks modelling the reader is referred to [13j. 
We suppose that the biological evolution is given by a family of genes Q = {gi : i = 1, N} interacting 
through a finite set of reactions 1Z. Every reaction r G 1Z can be represented as 

k 

ol\9x + a r 2 92 + ••• + a r N g N $ x g x + ... + j3 r N g N 

and it specifies that a\ molecules of i type (with 1 < i < N) called reactants interact in order 
to form the products (/3[ molecules of i type, with 1 < i < N). The reaction does not occur 
instantaneously and one needs to specify the reaction speed k r > 0. Also, the presence of all species 
is not required (a[, /3[ G N, for all 1 < i < N). The species are partitioned in two classes called 
continuous, respectively discrete component. This partition (for further considerations, see |13j ) 
induces a partition of the reactions. In sum, we distinguish between reactions contributing to 
the continuous flow (C = {1, 2, Mi}) and jump reactions {J = {M\ + 1, card (TZ)}). To every 
reaction r G 1Z, one associates 
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1) a stoichiometric column vector 9 r = f3 r — a r £ 

2) a propensity function A r : 1^ — > R + . 

N a r 

For a C-type reaction, A r (x) = k r Y[ x i 1 , for all x 6 R^. 

i=i 

For a jT-type reaction, one should require further regularity as Xi —> 0. The jump mechanism 
will specify that the number of molecules of type i diminishes by a\. Therefore, in order to insure 
positive components, rather then introducing A r (x) as for continuous reactions, one could consider 

A r (x) = k r J] X 

i=i \ a i 

aV>0 

for some regular function % such that < x < 1, x (y) = 0, for < y < 1 and x(v) = 1> f° r 
y > 1 + err (where err is a positive constant). 

The next step consists in the construction of two matrix M\ whose columns are the vectors a r , 
where r £ C, respectively M2 whose columns are the vectors a r , where r G J . The flow is the given 
by 

/ (x) = Mi x (Ai (x) , A 2 (x) , X Ml (x)) , 

the jump intensity 

A (») = E A r (x) 

reJ 

and, whenever A (x) > 0, the transition measure Q is given by 

Q (x, cfe) = T7T^+" r • 
reJ A K x ) 

One can suppose that all A r are bounded by a reasonable constant A max > 0, by replacing A r (x) 
by A r (x) A A max . Then, it is obvious that Al and A2 hold true. If h € BUC (R N ) , 

A (x + e) / h (z — e) Q (x + e, dz) — A (y + e) \ h (z — e) Q (y + e, dz) 

< ^ | Ar(x + e )/ l ( a; + e r)_ Ar ( y + e )^(j / + e r ) |< c( | a ._ J/ | +Wh( | a ._ 1/ | ))) 
rGj" 

for all x, y G M^, where c depends on the Lipschitz constant of A r , A max and WhW^ and Uh is the 
continuity modulus of h. This implies that B' (and a fortiori A3') hold true. The assumption A4 is 
a simple consequence of the fact that Q (x, •) is a probability measure for every x € R N . Also, one 
easily notices that 

Q ^x,R N \ B (x, sup |6> r |^ =0, 

which implies A5. It follows that all the assumptions we have made throughout the paper are 
naturally satisfied for piecewise deterministic systems associated to regulatory gene networks. 



4.2 On/Off Model 

A two-state model is often employed to describe different situations in the molecular biology. Usu- 
ally, the two states describe either the presence or the absence of some rare molecular specie. 
Whenever the gene 7 is inactive (represented by 7 = 0), the molecule X degrades at a rate ro, 
whileas, whenever 7 is active (7 = 1), the molecule X increases at a rate proportional to some 
given 77. 
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From the mathematical point of view, the system will be given by a process (X(t),j(t)) on the 
state space E = R x {0; 1}. The component X(t) follows a differential dynamic depending on the 
hidden variable 

dX = f -r (X), ifj(t) = 0, 

dt \ n(x), if 7 (t) = l, 

where tq(x) > is a bounded, Lipschitz-continuous consumption term and 77 (x) > is a bounded, 
Lipschitz continuous production term. To be more precise, the PDMP associated to the model has 
the characteristic (/, X,Q) given by / 7 (x) = — tq (x) (1 — 7) + 77 (x) 7, A 7 (x) = A 7 , Q( 7 , x; A) = 
Q(( 7 ,x); A) = £((i_ 7 ) )X )0A), for all 7 G {0, 1} , x G R, and all A C R. The vector field for the 
7 component can be considered to be 0. One should expect 0-consumption whenever X = and 
7 = i.e. ro(0) = 0, and no production whenever X = a max (some maximum level) and 7 = 1, i.e. 
ri(ot ma , x ) = 0. The assumptions A1-A5 are obviously satisfied. 

Proposition 18 The set K = [0, a max ] x {0, 1} is invariant with respect to the PDP associated to 
the On/Off Model. 

Proof. If x G [0 

j cimax] > then, by the definition of Q, 

Q((0,x),K c ) = Q((l,x),K c ) = 0. 
One notices that iV[ 0i(W ] (0) = R_ and iV[ 0itw ] (a ma x) = R+. For every p < 0, 

Pfo (0) = -pro (0) = and pf x (0) < 0. 

For every p > 0, 

p/o (a ma x) = -pro (a max ) < and pft (a max ) = 0. 
Thus, by applying Theorem [9J one gets the invariance of K. ■ 

Remark 19 The arguments of the previous Proposition yield that [a, b] x {0, 1} is invariant if and 
only if 

ro(a) = n(b) = 0. 

Therefore, in order for a point x$ to be a stable point, one should find a sequence e \ such that 
r o{xo — e) = ri(xo + e) = 0. In particular, a necessary condition is that ro(xo) = ^i(xo) = 0. 

We now focus on the model introduced in [T2] for stochastic gene expression and its implications 
on haploinsufficiency. This basic model of gene expression, product accumulation and product 
degradation can be given by the following reaction system: 



G 




G* 


Jp 


P 



This model considers a gene to switch randomly between inactive state (G) and active state (G*). 
The activation (respectively deactivation) rate is denoted by k a (respectively kd)- When active, each 
gene expresses a product (P) at a rate J p . The product is degraded at rate k p . One can represent 
this model as a particular case of the On/Off system by considering 

(44) r (x) = k p x, n(x) = J p - k p x, A = k a , X% = k d , and a max = 

The following result is a consequence of Criterion 1 17t 

Proposition 20 For every real constants a, b such that < a < b < a m ax 5 we let O = (a, b) x {0, 1} . 
Then, for every x G (0, a max ) , the set O is reachable with respect to the PDMP associated to Cook's 
model starting from (0, x) . 
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Proof. Let us fix x G (0, a max ). One seeks to apply Criterion [TTJ We reason by contradiction and 
assume, that, for every n G N*, there exists (p n ,ijj n G C\ (H.) such that 



(45) 



-n 1 < -<ff n (y) k p y + k a ip n (y) - (1 + fc a ) (y) - d[o,a]u[6,a max ] (y) + (x) , 
-n" 1 < ij)' n (y) (J p - k p y) - (1 + fc d ) ip n (y) + fc d y> n (y) - d[ ,a]u[6,<w] (y) + 0) 



l + k a -k 



I' 



for all y G [0, 1] . We multiply the first inequality by y fc p and integrate on (0, z] , for z > 0, to 
get 

l + fc a 1 + fcq 1 + fcq-fcp ft 1+fcq-fcp 

kpZ k p <p n (z) < p z k p (n' 1 + ip n (x))+k a / y fc p ip n (y)dy- y k p d[ 0>a ] U r bia i (y) dy, 

1 + K a JO JO 

or again 

1 + fcq — fcp 1 + fcq — fcp 

1 i" K a K p z k p K p z k p 

1+fcq-fcp 

for all G (0, a max ] • We multiply the second inequality in ([15]) by ( J p — k p y) k p and integrate 
on [z, a max ) , for z < a max , to get 

1 + fcq-fcp 

lpn(z) < J^TTtO 1 + i Pn{X))+k d 

(Jp k p z) kp 

1+fcq-fcp 

(A7 s _ J 2 - fc P y) fcp ^[o,a]u[b,« max ] (y) rfy 

i 4 'J 1 + fc^ ! 

( J p - k p z) k p 

for all z G [0, a max ) . We denote by a n the maximum value of (p n on [0, a max ] • It follows that 

i+k d -k p 

^n^)<T— r-(n + <p n (x)) + — — a„ ^ , 

i + i + (J p -k p z)~ 

for all z G [0, a max ) . We substitute the last inequality in (|35]L to have 

(48) ^ (z) " (i + i I + (n " + Vn (x)) + (ITWTy a " " 

for all G (0, a max ) . The function / is defined by 

i+k d -k p 



/(*) 



Jo I y P «[0,a]U[Mmax] W + K a2/ P i+fc d 

1 \ (Jp-M)~^ 

T. l + fcq 

hp „ fcp 



Jy 



for all z G (0, a max ) . One notices that 

k a f b i+k d -k P 

Um /(z) = / ( J P ~ k P U ) kp ^[0,a]U[6,a max ] («) ^ > 0. 

(l + fc a ) j/ P a 

Therefore, / (z) > (5, for some positive constant 5. In particular, the inequality (|48p written for 
z = x gives 

, x ^ fea + fed + 1 -1 , (1 + fco) (1 + k d ) 
<Pn [x) < — n + a n — 6. 
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We return to (|48p to obtain 



(49) 



/ \ ^ k a + kd + 1 -1 

<Pn [zj < — n + a n 



;i + ka) (1 + k d ) 



s, 



for all z G (0, a max ) • Finally, taking the supremum over z £ (0, a max ) , yields 

A; a + A; d + 1 



< S< 



-n 



(l + k a )(l + k d ) 

The last inequality fails to hold for large enough n. The assertion of our Proposition follows. ■ 

Remark 21 The reachability result is also true when starting from a generic point (l,x) replacing 
(0,x). 

We now illustrate (Figure 1) the viability result in Proposition 1181 and the reachability properties 
given by Proposition 1201 We use the classical description of the PDMP associated to Cook's model. 
The invariant set is represented in green ([0,a max ]) and we simulate a trajectory starting from a 
randomly chosen initial value for the protein. The time horizon is chosen very small (100) and the 
trajectory is represented in red. The reachable set is given by randomly generated a, b G (0, a max ) 
and is represented by the blue border lines. Whenever the sample remains in the target set for two 
consecutive time steps, the trajectory is represented in blue. 
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Figure 1. Invariance and reachability properties for Cook's model 



4.3 Bacteriophage A 

We consider the model introduced in [17 \ to describe the regulation of gene expression. The model 
is derived from the promoter region of bacteriophage A. The simplification proposed by the authors 
of [T7] consists in considering a mutant system in which only two operator sites (known as OR2 
and OR3) are present. The gene cl expresses repressor (CI), which dimerizes and binds to the 
DNA as a transcription factor in one of the two available sites. The site OR2 leads to enhanced 
transcription, while OR3 represses transcription. Using the notations in [T7], we let X stand for 
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the repressor, X2 for the dimer, D for the DNA promoter site, DX2 for the binding to the OR2 
site, DX% for the binding to the OR3 site and DX2X2 for the binding to both sites. We also denote 
by P the RNA polymerase concentration and by n the number of proteins per mRNA transcript. 
The dimerization, binding, transcription and degradation reactions are summarized by 







2X 






K 2 


D + X 2 


^DX 2 , 




K 3 


D + X 2 


*± DX*, 




K4, 


DX 2 + X 2 


+± DX 2 X. 


DX 2 + P 


^ DX 2 +P + nX 


X 





To this biological system we associate a piecewise deterministic process on the state space 
E = € {0, l} 4 : Ylt=i u i = l} x The characteristic is given by 

fu (xi,x 2 ) = f (xi,x 2 ) = (-2/ciXi 2 - k d xi + 2fc_ix 2 , kix\ - fc_ix 2 ) , 

A (u, x) = k 2 X 2 X (^2) V\ + k S X2X (^2) V\ + k 4 X 2 X (X2) V2 + k t V2 + fc-2^2 + ^-3^3 + 

A (u, x) Q ((A, x) ; dz) = k 2 x 2 x (^2) ^iS( XUX2 -i tV1 -i, U2 +i,u 3 ,^) (dz) 

+ k 4 X 2 X {X 2 ) t'2^(xi,X2-l,I'l,IA J -l,I'3,f4+l) 

+ ^ c -3 l/ 3^(2:i,X2+l, vi+l,v 2 , U3-I, (dz) + fc_4^'4<5(,j. 1) ,j; 2+ i )J , lil , 2+ i )i , 3il , 4 _i) 

for every (v,x) £ E. The function x is a smooth function such that < x < 1) X (y) = fo r 
y < 1 and x (?/) = 1 for ?/ > 1 + err. We consider the stability question for this system. Obviously, 
whenever a point (i/°, x°) is candidate to stability, one should at least expect that this point should 
be stable with respect to the deterministic evolution. One easily notices that the unique equilibrium 
point for the ordinary equation driven by the vector field / must satisfy x° = (0, 0) . Therefore, 
any candidate for stability with respect to the piecewise deterministic evolution associated to the 
lambda phage should be of this form. 

We shall prove that any small enough region surrounding (0, 0, 1, 0, 0, 0) is invariant with respect 
to the PDMP. We emphasize that similar arguments can be used to infer that no other point has 
similar stability properties. However, different invariant set may exist. Thus, bistability of bacte- 
riophage A should be understood as: a stable state (0,0,1,0,0,0) and some stability (invariance) 
region. 

The main result of the subsection is 



Proposition 22 For every Aki %_ 1 A 1 > e > 0, the set 

K £ = {(1,0,0,0)} x 



0, — — e 

k d 



x [0,e] 



is invariant with respect to the PDMP associated to the bacteriophage A model. 



Proof. We notice that, for every (xi,x 2 ) £ 







x [0, e] , one has 



A (1,0, 0,0, xi, x 2 ) = 0. 
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The following table gives, for all the possible values of (xi, X2) , the explicit form of the normal cone 
to K £ at (2:1,2:2) and the expression of (p, f (2:1,2:2)) for every p = (p%,p2) G N/^ e (2:1,2:2) ■ 



X\ 



0, 



2fc_i 







2fc_i 



2fc_i 



2k^ 



X 2 



(0,e) 



(0,e) 



(0,0 



x{0} 



{0} x M_ 



{0} x {0} 



{0}x 



x{0} 



(p,f(x l ,x 2 )) 







2p 1 k-ix 2 



(2pi -p 2 ) k-\e 



p 2 k\x\ 
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The last column allows to conclude that (p, f (x±, x 2 )) < 0, for all (2:1,2:2) G 
and all p G (xi, 2:2) . The conclusion follows from Theorem [9l ■ 







x [0,e] 



We illustrate the invariance result from the previous Proposition. The reader is invited to 
notice that, in the setting of the previous Proposition, the trajectory should be purely deterministic 



(A (1, 0, 0, 0, x\, X2) = 0, for every (xi, x 2 ) G 







e < 4fc jjj - A 1 and a starting point (2:1,2:2) G 



x [0, £■]). We randomly simulate the parameter 
x [0, e] . The simulated trajectory is 



represented in red and the bounds 2 \ 1 e and e are given in green. 
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Figure 2. Invariance properties for Bacteriophage A 
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5 Appendix 
5.1 Al 

We begin by sketching the proof of the comparison principle. 

Proof, (of Proposition [6|) Let us suppose that, for some positive constant 6 > 0, 



(50) 

For every 5 > 0, we define 



sup (W(x) - V(x)) = > 0. 



sup (w(x) - V(x) -S\x\ 2 ) . 



It is obvious that (6s) is decreasing and lim^o 9s = 0. For every e > 0, we introduce 

2 



(51) 



$ E ,s(x,y) = W(x)-V(y) 



x-y 



s 



o \x\ , 



for all x,y 6 R . We recall that W and V are bounded. Then, using the u.s.c. of & £ ,8-, we get the 
existence of some global maximum point (x £ ^,y £ ,8) € M. 2N of & £ ,s- Standard arguments yield the 
existence of some x$ G W N such that #5 = W (xs) — V (xs) — 5\xs\ 2 , and 



(52) 

We also obtain 
(53) 



(i) lim e ^o 



x s,s-y £ ,s 



0, (ii) l\m e ^ x £ j = lim e ^oye,S = x s , 



(Hi) lim e ^ W (x eS ) = W (x s ) , (iv) lim £ _^ V (y £)< s) = V (x s ) ■ 



lim 5 \xs\ = 0. 
<5^0 



We recall that W is a viscosity subsolution and consider the test function 

2 

r — //. . 

<Pe,s( x ) = V (Ve,s) + 



x - y £ ,s 



£ 



+ S \x\ 



for all iEl' v . We get 



> W (x E>S ) - d K [x £) s) A 1 + SUp <^ J (/ ( X £,<5) n ) > x e,<5 _ Vefi) 

- 25 (/ (a; e a, u) , x e S ) - A (x e u) / (2) - W (x e5 )) Q (x E> s, u, dz) 

> W (x EjS ) - d K (x £) s) M-CS \x £)S \ + sup { -\ (f (x EjS , u) , x E>s - y £ ,s) 

ueu I £ 

- A (x E}S , u) / (W (z) - W (x eS )) Q (x EjS , u, dz) \ , 

Jr n ) 

where C is a generic real constant that may change from one line to another. Standard estimates 
yield 

> W (x e x) - d K (x e *) A 1 - C(S \x e *\ + \W (x e x) - W (x s )\ + \x e * - x s \) 



(54) 



+ sup i — 2 (/ (x e S , u) , x £)S - y e ,s) 
- A (x s , u) [ (W (z) - W (x s )) Q (x E , s , u, dz) \ . 
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In a similar way, one has 



< V (y £ ,g) ~ d K (y £> g) M + C(\V (y e>s ) - V (x 5 )\ + \y s>s - x s \) 



(55) + sup <^ — 2 (/ (Ve,S, u ) , x e,S - Ve,s) - A (x s , u) / (V (z) - V (x s )) Q (y £ g, u, dz) 
ueu L £ ' ' 



Combining (|54|) and (|55j) . we get 

(56) 

< V (y £ , s ) - W (x £ ,g) + C 



X £ ,S ~ Ve,6 



£ 



+ \x £t g - xg\ + \y £j g - xs\ + J 
+ C(5 \x £ ,g\ + \V (y £ ,g) - V (x s )\ + \W (x £ ,g) - W (x s )\) 

+ sup <^ A (x s , u)[ (W (z) - W (x s )) Q (x £ s, u, dz) - I (V(z)-V (x s )) Q {y £ ,s, u, dz) 
On the other hand, we notice that 



(W (z) - W (x s )) Q (x £ , s , u, dz) - (V(z)-V (x s )) Q (y £ ,s, u, dz) 

JR" 

< f (W(z)-V(z)-W(x s ) + V(x s ))Q(x £ ,s,u,dz) 
Jr n 

+ / V (z)(Q(x £ S ,u,dz) - Q(y £>5 ,u,dz)) . 



Thus, whenever V is continuous, 



(W (z) - W (xs)) Q (x £ , 5 , u, dz) - (V(z)-V (xs)) Q (y £ , s , «, dz) 

Jr n 

(57) < J ^5\z\ 2 AC|q (xs,u,dz) +V(s\-\ 2 aC) O^M ~ + W (\x E ,s - y e ,s\) , 

where rjs and r]y are given by Assumption A3 and are independent of u € U. Similar estimates 
hold true if W is continuous. We substitute ([5"7|) in (|56j) and take limsup as e — > in ([56]) to obtain 



< — 6s + 5 \xg\ + C sup / ( 5 \z\ 2 A 1 ) Q (xs, u, dz) 
ueu Jrn V / 



< — 6s + S \xg\ + C sup < /_ 5\z\ Q (xg,u,dz) + Q [xg,u, 

ueu I Jb(x s ,s-^ ' 



i N \ B ( x 5 , <Tt 



l N n BtxJ-t 



< — 6s + C ( 5 \xs\ 2 + 5 \xg\ + 5? j + sup Q (x,u,l 

^ ' (x,u)eR N xU ^ 

We allow 5 —> in the last inequality and recall that A5 holds true to have 

< -6. 

This comes in contradiction with (|50p . The proof is now complete. ■ 
5.2 A2 

The Proof of Theorem [16] relies on the fact that the functions V s defined by (|42p are viscos- 
ity subsolutions of the Hamilton- Jacobi integro-differential equation (|27p . The proof adapts the 
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arguments used in Barles, Jakobsen [9j Lemma 2.7. Following the proof of this Lemma, we in- 
troduce, for every h > 0, u 2 G R N , Qf = u 2 + [~\,\) N , p^ = L K 2 p £ (y)dy, and 4 (x) = 

^ u 2 gftZ jv jOe' w f e (x — u 2 ) • Thus, Ih is a convex combination of bounded, uniformly continuous vis- 



cosity subsolutions of (J27J). Moreover, by classical results, the discretization Ih converges uniformly 
to V s . To conclude, we show that viscosity subsolutions are preserved by convex combination and 
uniform convergence. 

Proposition 23 Given two bounded, uniformly continuous viscosity subsolutions v\ and v% of the 
Equation (21) and two nonnegative real constants Ai, A2 G R+ such that Ai + A2 = 1, the convex 
combination X\Vi + X 2 v 2 is still a viscosity subsolution of \2T\ ). 

Proof. The assertion is trivial when either Ai = or A2 = 0. If A1A2 7^ 0, we let x £ R N and 
((JGCj 1 (Mx) be a test function such that 

(58) A1U1 (x) + X 2 v 2 (x) -<p(x)> X\vi (y) + X 2 v 2 (y) -tp(y), 

for all y G R N . We may assume, without loss of generality that tp G Cb (R N ) . Indeed, whenever <p 
does not satisfy this assumption, one can replace it with some ip° defined as follows : First, notice 
that there exists some r > such that B (x, 2r) C Mx- We define 

tp° (y) = (<p (y) + Aiui (x) + x 2 v 2 (x) - if (x)) x (y) 
+ (Ai«i [y) + X 2 v 2 (y))(l-x(y)), 

for all y G M. N , where x is a smooth function such that < x < 1) = 1, if 2/ G -B (x, r) and 
x(y) = 0, if y G \ -B (x, 2r). Then (|58p holds true with ip° instead of ip. The new function ip° 
also satisfies 

Vv?° (x) = Vp (x) . 

We introduce, for every e > 

$ e (x, y) = Xivi (x) + A 2 t> 2 (y) ~ X\<p (x) - X 2 ip (y) - \ |x - y\ 2 - \x - x| 2 , 

for all x, y G M . We recall that the functions v\ } v 2 and </? are bounded and continuous. This yields 
the existence of a global maximum (x £ ,y £ ) of <£ e . Moreover, by standard arguments, 



(59) lim x e = lim y e = x, lim 

We consider the test function ip given by 



Ve 



2 

= 0. 



A: 



-1 



i> ( x ) = -^2\ 1 V2 (ye) + ^ (x) + X^ V (ye) + |z ~ Vef + A x 1 |x - x| 2 , 

for all x G M^. We recall that the function v\ is a viscosity subsolution for (|27p . Then, 

/ 2A _1 \ 

vi (x e ) + d c (x £ ) A 1 + H lx £ , Vip (x £ ) + — i- (x £ - y e ) + 2A7/ 1 (x £ - x) , v x J < 0. 

Standard estimates yield 

> vi (x) + cfce (x) A 1 + sup { - (/ (x, u) , V99 (x)) -jjj- (x £ - j/ e , / (x e , u)) 



— A (x, u) \ (vx (z) — vi (x)) Q (x,u,dz) 
(60) - C (\x £ - x| + |ui (x e ) - ui (x)| + |Vv? (x £ ) - V<p (x)| + ^ (|x £ - x|)) . 
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In a similar way, we get 

> v 2 (x) + d c (i)M + sup <^ - (/ (x, u) , Vip (x)) H §- (x £ - y £ , f (y e ,u)) 

-X(x,u) / (v 2 (z) - v 2 (x))Q (x,u,dz) 

(61) - C (|y e - 2c| + |«2 (ife) - ^2 (ac)| + |Vy> (y e ) - Vip (x)\ + r, V2 (\y £ - x\)) 

Finally, using (f60|) . ([6T]) and ([59]) . and passing to the limit as e — > 0, yields 

(Ai«i + A2V2) (sc) + d c (x) A 1 + H (x, Vip (x) , Xivi + X 2 v 2 ) < 0. 

■ 

These arguments allow to obtain, by recurrence, that any convex combination of continuous, 
bounded viscosity subsolutions is still a subsolution for (j2?j) . 



Proposition 24 (Stability) 

Let (v n ) n be a sequence of continuous, uniformly bounded viscosity subsolutions of (27\ ). More- 
over, we suppose that v n converges uniformly on compact sets to some continuous, bounded function 
v. Then the function v is a viscosity subsolution of \21\) . 

Proof. We let x 6 R N and (p £Cj (Af x ) be a test function such that v — <p has a global maximum 
at x. As in the previous proposition, one can assume, without loss of generality, that ip £ Cj 
Classical arguments yield the existence of some point x n G R N such that 



for all y <ER N and 



v n (x n ) - V (x n ) ~ \x n ~ x\ 2 > v n (y) - ip (y) - \y - x\ 2 , 



lim x n = x. 

n— >oo 



We assume, without loss of generality, that \x n — x\ < 1, and x n G Af x , for all n > 1. Then, 

- (/ (x n ,u) , Vip (x n ) + 2(x n - x)) 



(62) > V n (x n ) + do? (X n ) A 1 + SUp . f , 

We have 
(63) 

- (/ (x n ,u) ,Vip(x n ) + 2{x n - x)) > - (/ (x,n) , Vip{x)) - C (\x n -x\ + \Vtp(x n ) - Vip(x)\) , 

where C > is a generic constant independent of n > 1 and u £ U which may change from one 
line to another. We also get 



-X(x n ,u) / (v n (z) -v n (x n ))Q(x n ,u,dz) 

JWL N 

> -X (x,u) / (v(z) - v (x)) Q (x,u,dz) - C (\x n - x\ + \v n (x n ) - v (x)\ + r/ v (\x n - x\)) 
(64) -Csup/ \v n (z) - v (z)\ Q (x n ,u,dz) . 



Finally, for every m > 1, 

sup / \v n (z) - v(z)\Q (x n , u, dz) 



u 



< sup (\v n (z) - v(z)\) + Csup Q (x n ,u,R N \ B (0, m + \x\ + 1)) 

z£~B(0,m+\x\ + l) u£U 

< sup i\vn{z) — v i z )\) + CsupQ (x n , u,R \ B (x n , m)) 

zeB(0,ro+|a:|+l) u&U 

(65) < _ sup (\v n (z) - v{z)\) + C sup g( 1 / ) «,R JV \5(y,m)). 

^eB(0,m+|x|+l) y&& N ,ueu 
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We substitute ([6"3"l)-([6l)D in (1521 and allow n — > oo to have 



> v (x) + do c (x) A 1 + sup < — (/ (x, u) , Vip (x)) — A (x, u) \ (v(z) — v (x)) Q (x, u, dz) 

ueu [ Jr n 



(66) -C sup Q(y,u,R N \B(y,m)) 

~~>,ueu 



for all m > 1. We conclude using the Assumption A5. ■ 
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